fix(runner): verify the datadir head for a bundle inside the fixtures artifact - #306
Merged
Merged
Conversation
… artifact verifyPreRunBundleHead guards the replay's by-number skip: a datadir at the bundle's height but on a different chain would otherwise skip every line and benchmark the wrong state. It only ran when pre_runs named a local_fixtures_dir, so since #296 — which let a bundle be resolved out of the extracted fixtures artifact instead — it silently no-opped for exactly the configs that rely on that path. Ask the source where its bundle is, via a new optional PreRunBundleLocator, rather than re-deriving the path from config: the bundle may sit at a local directory or inside the artifact, and only the source knows which. EESTSource now resolves it once, for both the replay and this check. The config-derived path stays as a fallback for callers without a live source. ReadPreRunBundleInfoAt reads a bundle directory named outright, which is what a runner-side pre_runs source has; ReadPreRunBundleInfo keeps its parent + PreRunBundleSubdir form. An unlocatable bundle now warns instead of passing quietly, since the skip it guards runs either way.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #305.
verifyPreRunBundleHeadexists to guard the replay's by-number skip — a datadir at the bundle's height but on a different chain would otherwise skip every line and benchmark the wrong state. It never ran for artifact-resolved bundles:Since #296 a
pre_runssource can resolve its bundle out of the extracted fixtures artifact using onlyfixtures_subdir, leavinglocal_fixtures_dirempty — so the guard returned early for exactly the configs that depend on that path. That gap came in with #296, which taughtloadPreRunBundleStepsthe fallback and left this second call site behind.Change
Ask the source where its bundle is instead of re-deriving the path from config. The bundle may sit at a configured local directory or inside the extracted artifact, and only the source knows which — a second config-derived derivation is what silently missed the artifact case.
EESTSource.PreRunBundleDir()resolves it once and is now the single source of truth;loadPreRunBundleStepsuses it too, so the replay and the check can no longer disagree about which bundle is in play.LocalSource,GitSourceandArchiveSourceare untouched.ReadPreRunBundleInfoAt(bundleDir)reads a directory named outright, which is what a runner-sidepre_runssource has.ReadPreRunBundleInfo(parent)keeps its parent +PreRunBundleSubdirform and simply delegates.No behaviour change for a
local_fixtures_dirconfig: it resolves to the same directory by either route.Why it matters
Two pre-runs of the same config produce the same end block number with different hashes:
Running the compute suite against stateful-promoted datadirs, the replay reported success while skipping everything, and then every benchmark payload was an orphan — reth stuck on
forkchoice updated message when syncing, nethermind0 passed of 244 attempted. Hours of wall-clock per client, and it reads as a client bug rather than a wrong datadir. With this change the run stops up front withthe datadir is on a different chain than the bundle was recorded against.Testing
TestVerifyPreRunBundleHeadFromFixturesArtifact— the regression:pre_runswith nolocal_fixtures_dir, bundle located via the source. Covers wrong-hash-now-rejected, matching-head-still-applied, and the no-locator/no-local-dir case. Onmasterthe first of these passes silently, because the early return means no verification happens at all.TestPreRunBundleDir— table-driven over local-wins, artifact fallback, default subdir, and both nothing-to-resolve cases, plus a compile-timePreRunBundleLocatorassertion.TestReadPreRunBundleInfoAt— reads a bundle dir directly and asserts it agrees with the parent-form call.TestVerifyPreRunBundleHeadsubtests still pass; the config-derived fallback is what keeps them green, and dropping it broke them, which is how I found it was load-bearing.gofmt,go vetandgo test -raceare clean forpkg/executor,pkg/builderandpkg/runner.golangci-lint run --new-from-rev=origin/masterreports 0 issues forpkg/executorandpkg/builder; it cannot loadpkg/runnerin my environment (pkg/podmanneeds cgogpgme, which also fails on pristinemasterhere), sopkg/runnerwas built, vetted and tested with-tags containers_image_openpgpinstead — CI will cover it properly.